Imaging device and portable information terminal

ABSTRACT

A solid-state imaging device according to an embodiment includes: an imaging element formed on a semiconductor substrate, and comprising an imaging region including a plurality of pixel blocks each including a plurality of pixels; a first optical system forming an image of an object on an imaging plane; and a second optical system comprising a microlens array including a plurality of microlenses each corresponding to one of the pixel blocks, and reducing and re-forming the image to be formed on the imaging plane on the pixel blocks corresponding to the respective microlenses. The imaging plane of the first optical system is located further away from the first optical system than the imaging element when the object is located at an infinite distance.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromprior Japanese Patent Application No. 2011-40254 filed on Feb. 25, 2011in Japan, the entire contents of which are incorporated herein byreference.

FIELD

Embodiments described herein relate generally to a solid-state imagingdevice that is capable of obtaining a two-dimensional visible image anddepth-direction distance information about the object at the same time,and to a portable information terminal.

BACKGROUND

Various techniques such as a technique using reference light and astereo distance measuring technique using two or more cameras have beensuggested as imaging techniques for obtaining information about thedistance to an object in the depth direction as two-dimensional arrayinformation. Particularly, in recent years, there has been an increasingdemand for relatively inexpensive products as novel consumer inputdevices.

In view of this, an imaging device that has a multi-view structure andincludes an imaging lens has been suggested as a structure for obtaininga large number of parallaxes from multiple viewpoints, and restraining adecrease in resolution. This imaging device has an imaging lens, forexample, and has a multiple optical system as an image re-formingoptical system located between the imaging lens and an imaging element.As the multiple optical system, a microlens array or the like having alarge number of minute lenses formed on a flat surface is used. Pixelsfor obtaining the images of the microlenses are provided atcorresponding locations below the respective microlenses. An imageformed at the imaging lens is re-formed on the imaging element throughthe image re-forming microlenses, and the re-formed single-view image isan image having a shift in viewpoint. The shift is equivalent to theparallaxes existing because of the locations of the microlenses.

By performing image processing on parallax images obtained through thelarge number of microlenses, the distance to the object can be estimatedby using the principle of triangulation. Further, by performing imageprocessing to combine such parallax images, the parallax images can bere-formed as a two-dimensional image.

In the above-described imaging device having a multi-view structure,however, the focal plane of the imaging lens is shifted forward (towardthe object) to a position before the imaging element when the object islocated at an infinite distance. Therefore, when the same imaging lensas above is used, camera height is increased.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a solid-state imaging device accordingto a first embodiment;

FIG. 2 is a cross-sectional view of the solid-state imaging device ofthe first embodiment;

FIGS. 3A and 3B are diagrams for explaining the relationship between thedistance from the imaging lens to an object and light rays in the firstembodiment;

FIGS. 4A and 4B are diagrams for explaining the geometric opticalrelationship of microlenses at the center of the optical axis of theimaging lens in the first embodiment;

FIGS. 5A through 5D are diagrams for explaining microlens imagesobtained by the optical system of the first embodiment and imagere-formation;

FIGS. 6A through 6C are diagrams for explaining the correspondencerelationship between the distance from the imaging lens to an object andmicrolens imaging magnification in the first embodiment;

FIGS. 7A through 7C are diagrams for explaining the relationship ofoverlapping fields of view among adjacent microlenses;

FIG. 8 is a cross-sectional view of a solid-state imaging deviceaccording to a second embodiment; and

FIG. 9 is a diagram showing an example of a portable informationterminal that uses the solid-state imaging device of the first or secondembodiment.

DETAILED DESCRIPTION

A solid-state imaging device according to an embodiment includes: animaging element formed on a semiconductor substrate, and comprising animaging region including a plurality of pixel blocks each including aplurality of pixels; a first optical system forming an image of anobject on an imaging plane; and a second optical system comprising amicrolens array including a plurality of microlenses each correspondingto one of the pixel blocks, and reducing and re-forming the image to beformed on the imaging plane on the pixel blocks corresponding to therespective microlenses. The imaging plane of the first optical system islocated further away from the first optical system than the imagingelement when the object is located at an infinite distance.

The following is a description of embodiments of the present invention,with reference to the accompanying drawings.

First Embodiment

FIG. 1 shows a solid-state imaging device (a camera module) 1 accordingto a first embodiment. The solid-state imaging device 1 of the firstembodiment includes an imaging module unit 10 and an image signalprocessor (hereinafter also referred to as ISP) 20.

The imaging module unit 10 includes an imaging optics 12, a microlensarray 14, an imaging element 16, and an imaging circuit 18. The imagingoptics 12 functions as an imaging optical system that captures lightfrom an object into the imaging element 16. The imaging element 16functions as an element that converts the light captured by the imagingoptics 12 to signal charge, and has pixels (such as photodiodes servingas photoelectric conversion elements) arranged in a two-dimensionalarray. The microlens array 14 is a microlens array that includesmicrolenses or a microoptical system including prisms, for example. Themicrolens array 14 functions as an optical system that reduces andre-forms a group of light rays formed on the imaging plane by theimaging optics 12, on pixel blocks corresponding to the respectivemicrolenses. The imaging circuit 18 includes a drive circuit component(not shown) that drives the respective pixels of the pixel array of theimaging element 16, and a pixel signal processing circuit component (notshown) that processes signals output from the pixel region. The drivecircuit component includes a vertical select circuit that sequentiallyselects pixels to be driven by the horizontal line (row) in the verticaldirection, a horizontal select circuit that sequentially selects pixelsby the column, and a TG (timing generator) circuit that drives thoseselect circuits with various pulses. The pixel signal processing circuitcomponent includes an AD converter circuit that converts analog electricsignals supplied from the pixel region into digital signals, a gainadjustment/amplifier circuit that performs gain adjustments andamplifying operations, and a digital signal processing circuit thatperforms corrections and the like on digital signals.

The ISP 20 includes a camera module interface 22, an image capturingunit 24, a signal processing unit 26, and a driver interface 28. A RAWimage obtained through an imaging operation performed by the imagingmodule unit 10 is captured from the camera module interface 22 into theimage capturing unit 24. The signal processing unit 26 performs signalprocessing on the RAW image captured into the image capturing unit 24.The driver interface 28 outputs the image signal subjected to the signalprocessing performed by the signal processing unit 26, to a displaydriver (not shown). The display driver displays the image formed by thesolid-state imaging device.

FIG. 2 is a cross-sectional view of the solid-state imaging device 1according to the first embodiment. As shown in FIG. 2, in thesolid-state imaging device 1 of the first embodiment, the imagingelement 16 is formed on a semiconductor substrate 16 a. Pixels 16 bincluding photodiodes are formed on the semiconductor substrate 16 a,and a drive/read circuit (not shown) that drives the pixels 16 b andreads signals from the pixels 16 b is also formed on the semiconductorsubstrate 16 a. Color filters 16 c of R (red), G (green), and B (blue)corresponding to the respective pixels 16 b are formed on the pixels 16b in an arrangement such as a Bayer arrangement. On the color filters 16c, microlenses 16 d for collecting pixel light may be formed for therespective pixels.

A visible light transmissive substrate 40 that has the microlens array14 attached thereto and transmits visible light is provided above thecolor filters 16 c. The visible light transmissive substrate 40 may havea multilayer film or a single-layer film that transmits visible lightbut reflects near-infrared light. The microlens array 14 includes amicrolens unit 14 a formed on a microlens array substrate 14 b. Themicrolens unit 14 a is placed on the side of the imaging element 16,when seen from the visible light transmissive substrate 40, and includesmicrolenses. The respective microlenses correspond to the pixel blocksformed by the pixels provided on the semiconductor substrate 16 a, andfunction as an optical system that reduces and forms an image on thecorresponding pixel blocks. The visible light transmissive substrate 40is also joined to the semiconductor substrate 16 a with a resin spacer42 provided around the imaging region having the pixels 16 b formedtherein. The positioning for joining the semiconductor substrate 16 aand the visible light transmissive substrate 40 to each other isperformed based on positioning marks or the like. The visible lighttransmissive substrate 40 may be made of a material that cuts offunnecessary near-infrared light or may have a film that cuts offnear-infrared light.

In the semiconductor substrate 16 a, read electrode pads 44 for thepixels 16 c are formed, and through electrodes 35 that penetrate throughthe semiconductor substrate 16 a are formed under the electrode pads 44.The semiconductor substrate 16 a is electrically connected to a chip 50via the through electrodes 46 and bumps 48. A driving/processing circuit(the imaging circuit 18) that drives the imaging device and processesread signals is formed in the chip 50.

The imaging lens 12 is provided above the visible light transmissivesubstrate 40. The imaging lens 12 is attached to a lens barrel 62, andthe lens barrel 62 is attached to a lens holder 64. The lens holder 64is joined to the peripheral region of the visible light transmissivesubstrate 40. When the imaging lens 12 is attached to the lens barrel62, the focal length of the lens 12 may be adjusted in accordance withthe relationship between the pressing force and the output image. Alight shielding cover 52 for shielding unnecessary light is attached tothe peripheries of the semiconductor substrate 16 a, the visible lighttransmissive substrate 40, and the chip 50. Module electrodes 54 thatelectrically connect the chip 50 and the outside are formed in the lightshielding cover 52. However, the present invention is not limited to theabove described structure, and the electrode pads 48 may be electricallyconnected to an external chip by wire bonding or the like.

(Relationship Between Object Distance and Light Rays)

Referring now to FIGS. 3A and 3B, the relationship between the distanceto an object and the group of light rays in an optical system (a virtualimage optical system) of the solid-state imaging device 1 of thisembodiment is described.

Normally, in a conventional solid-state imaging device (a camera module)of a conventional fixed focus type, the distance between the image-sideprincipal surface of the imaging lens and the imaging element is madenearly equal to the focal length f of the imaging optics. Accordingly,the distance between a photographable object and the camera is in therange of the shortest image pick-up distance at which image blurring ison the highest allowable level, to an infinite distance.

In the solid-state imaging device 1 of this embodiment, on the otherhand, the imaging element 16 is positioned so that the distance betweenthe image-side principal surface 12 a of the imaging lens 12 and theimaging element 16 becomes shorter than the focal length f of theimaging optics (the imaging lens 12), as shown in FIG. 3A. That is, whenthe microlens array 14 is not provided, an imaging plane 70 is locatedbehind the imaging element 16 when the object is at an infinitedistance, and a focused image cannot be picked up by the imaging element16. The imaging plane 70 when the microlens array 14 is not provided iscalled a virtual imaging plane. In this embodiment, however, themicrolens array 14 is provided between the imaging lens 12 and theimaging element 16, so that the light ray group consisting of aprincipal ray 80 and peripheral rays 82 is refracted, and the light raygroup forms an image on the surface of the imaging element 16 locatedbefore the virtual imaging plane 70. At this point, the image to beformed on the virtual imaging plane 70 by the imaging lens 12 is reducedand picked up by the microlenses 14 a. The reduction rate serves as theindicator of degradation of the optical resolution, but this will bedescribed later.

FIG. 3B shows the positional relationship between the imaging lens 12and the virtual imaging plane 70 when an object 100 is located at afinite distance A. As shown in FIG. 3B, when the object 100 is at ashorter distance than an infinite distance, the virtual imaging plane 70is located even further behind the imaging element 16 than when theobject 100 is located at the infinite distance. Even in such a case, thelight ray group to form an image on the virtual imaging plane 70 formsan image on the surface of the imaging element 16 through themicrolenses 14 a. In FIGS. 3A and 3B, LML represents the pitch of themicrolenses 14 a.

The distance D at which the microlenses 14 a perform image re-formationvaries between the case where the object 100 is located at the infinitedistance and a case where the object 100 is located at the finitedistance. However, the variation is sufficiently smaller than thevariation of the distance A from the object 100 to the imaging lens 12,and therefore, the variation of the distance D is regarded as beingwithin the depth of focus.

As described above, the virtual imaging plane 70 of the imaging lens 12is located behind the imaging element 16, so that the distance betweenthe imaging lens 12 and the imaging element 16 can be made shorter thanthat in the conventional case where the virtual imaging plane 70 islocated before the microlens array 14. Accordingly, the height of acamera module can be made smaller when an imaging lens having the sameoptical properties as above is used.

Further, the distance between the imaging lens 12 and the imagingelement 16 has a smaller designed value than that in a conventionalcamera optical system having no microlenses.

Accordingly, the height of a camera module can be made smaller when animaging lens having the same optical properties as above is used.

(Relational Expression of Geometric Optics in Solid-State Imaging Deviceof First Embodiment)

Referring now to FIGS. 3A, 3B, 4A, and 4B, the relationship of thegeometric optics in the optical system (the virtual optical system) ofthe solid-state imaging device 1 of this embodiment is described. Forease of explanation, only a paraxial region is described herein. Itshould be noted that FIG. 4B is an enlarged view of the paraxial regionof the microlenses shown in FIG. 4A.

Where attention is paid only to the imaging optics (the imaging lens)12, the principal ray 80 and its peripheral rays 82 from the object 100forms an image on the virtual imaging plane 70 determined from the focallength f of the imaging optics 12 and the distance between the imagingoptics 12 and the object 100, so that the following relationshipexpressed by the following equation (1) is satisfied:

$\begin{matrix}{\frac{1}{f} + \frac{1}{A} + \frac{1}{B}} & (1)\end{matrix}$

where f represents the focal length of the imaging lens 12, A representsthe distance from the object-side principal surface 12 b of the imaginglens 12 to the object 100, and B represents the distance from theimage-side principal surface 12 a of the imaging lens 12 to the virtualimaging plane 70. The imaging magnification (the lateral magnification)of the imaging lens 12 is expressed by the following equation:

$\begin{matrix}{M = \frac{B}{A}} & (2)\end{matrix}$

In this embodiment, the virtual imaging plane 70 of the imaging lens 12is located further behind the imaging element 16 (on the opposite sideof the imaging element 16 from the object 100). Since the microlenses 14a is positioned before the virtual imaging plane 70, light is collectedonto the surface of the imaging element 16 that is located before thevirtual imaging plane 70 and has pixels formed thereon. At this point,the light rays 80 and 82 form a reduced image as a virtual image. Theimaging system of the microlenses 14 a is expressed by the followingequation:

$\begin{matrix}{\frac{1}{g} = {{- \frac{1}{C}} + \frac{1}{D}}} & (3)\end{matrix}$

where g represents the focal length of the microlenses 14 a, Crepresents the distance from the object-side principal surface of themicrolenses 14 a to the virtual imaging plane 70, and D represents thedistance from the image-side principal plane of the microlenses 14 a tothe imaging element 16. At this point, the imaging magnification of theimaging system of the microlenses 14 a is expressed by the followingequation (4):

$\begin{matrix}{N = \frac{D}{C}} & (4)\end{matrix}$

The variable E of the following equation (5) is then introducedaccording to a geometric relationship. In a case where the opticalsystem is a fixed-focus optical system, the variable E is a fixeddesigned value.

E=B−C  (5)

When two adjacent microlenses 14 a are selected, the array pitch of themicrolenses 14 a or the distance between the microlenses 14 a isrepresented by LML. Light rays 84 a, 84 b, 84 c, and 86 emitted from thesame object form an image on adjacent respective microlenses 14 a. Thepitch LML and the shift length A on one side are expressed by thefollowing equation (6) according to the geometric relationships of themain light rays 84 a, 84 b, and 84 c with respect to the respectivemicrolenses 14 a shown in FIG. 4B:

$\begin{matrix}{\frac{C}{L_{ML}} = \frac{D}{\Delta}} & (6)\end{matrix}$

Using the above parameters, the variations of the respective parameters(B, C, D, and Δ) with respect to shifting of the object (a variation ofA) are described. The distance from the imaging lens 12 to aphotographable object is represented by A0, and parameters having thesubscript “0” on the lower right sides thereof (B0, C0, D0, and Δ0)represent the values obtained when the distance from the imaging lens 12to the object is A0. Once A0 is determined, the above parameters areuniquely determined at the same time as A0, if the optical system is afixed-focus optical system.

The variation of the parameter D obtained when the distance from theimaging lens 12 to the object is changed from A0 to A is presented by M(the imaging lens magnification). According to the equations (1) through(5), the imaging lens magnification M satisfies the relationshipexpressed by the following equation (7):

$\begin{matrix}{D = {\left( {\frac{1}{g} + \frac{1}{C}} \right)^{- 1} = {\left( {\frac{1}{g} + \frac{1}{\left( {B - E} \right)}} \right)^{- 1} = {\left( {\frac{1}{g} + \frac{1}{B - \left( {B_{0} - C_{0}} \right)}} \right)^{- 1} = \left( {\frac{1}{g} - \frac{1}{{- \frac{D_{0}}{N_{0}}} + {\left( {M_{0} - M} \right)f}}} \right)^{- 1}}}}} & (7)\end{matrix}$

According to the equations (1), (2), (6), and (7), the shift length Δbetween the distance A from the imaging lens 12 to the object and theimage satisfies the relationship expressed by the following equation(8):

$\begin{matrix}{A = {\left( {\frac{1}{f} - \frac{1}{B}} \right)^{- 1} = {\left( {\frac{1}{f} - \frac{1}{B_{0} - C_{0} + C}} \right)^{- 1} = \left( {\frac{1}{f} - \frac{1}{{\left( {M_{0} + 1} \right)f} - \frac{D_{0}}{N_{0}} + \frac{L_{ML}D}{\Delta}}} \right)^{- 1}}}} & (8)\end{matrix}$

The shift length Δ and the magnification M satisfy the relationshipexpressed by the following equation (9):

$\begin{matrix}{\Delta = \frac{{DL}_{ML}}{{\left( {M_{0} - M} \right)f} - \frac{D_{0}}{N_{0}}}} & (9)\end{matrix}$

As the distance A to the object becomes longer or as the distance Abecomes closer to ∞, the magnification M becomes closer to 0, and theshift length Δ approaches the value expressed by the following equation(10):

$\begin{matrix}{\Delta = \frac{{DL}_{ML}}{{M_{0}f} - \frac{D_{0}}{N_{0}}}} & (10)\end{matrix}$

Here, the variation of A with respect to the variation of A is expressedby the following equation:

$\begin{matrix}{\frac{\partial\Delta}{\partial A} = {{\frac{\partial\Delta}{\partial M}\frac{\partial M}{\partial A}} = {{\frac{{DL}_{ML}f}{\left\lbrack {{\left( {M_{0} - M} \right)f} - \frac{D_{0}}{N_{0}}} \right\rbrack^{2}} \times \left( {- \frac{M^{2}}{f}} \right)} = \frac{{- {DL}_{ML}}M^{2}}{\left\lbrack {{\left( {M_{0} - M} \right)f} - \frac{D_{0}}{N_{0}}} \right\rbrack^{2}}}}} & (11)\end{matrix}$

Since the equation (11) indicating accuracy includes M (=B/A), theaccuracy has distance dependence.

(Relationship Between Microlens Images Obtained by Optical System ofFirst Embodiment and Image Re-Formation)

Referring now to FIGS. 5A through 5D, the optical image relationshipobtained by the optical system of the first embodiment is described.

In a case where microlenses are not provided in the optical system, andthe imaging element 16 is located on the virtual imaging plane 70, animage of an object should be formed on the virtual imaging plane 70, asshown in FIG. 5C. On the other hand, when light rays emitted from thesame object are divided by the microlenses 14 a to form an image on thesurface of the imaging element 16 located before the virtual imagingplane 70 as in this embodiment, two or more images of the same objectare captured in accordance with parallaxes caused within the lensaperture. Therefore, images of the same object picked up by themicrolenses are output as an image, as shown in FIG. 5B.

The microlens images shown as an example in FIG. 5B indicate an imageobtained where uniform-pitch microlenses are arranged in a hexagonalclose-packed array. The microlens images shown in FIG. 5B are reduced insize from the optical image to be formed on the virtual imaging plane70, at the imaging magnification N (the equation (4)) of the microlensimaging system. The size reduction rate N serves as the indicator ofdegradation of the optical resolution, but this will be described later.

The obtained microlens images are re-formed as a two-dimensional imagehaving no overlapping portions as shown in FIG. 5D, by performing are-forming operation on read image data of the respective microlenses.The re-forming operation will be described later.

Having the parallaxes within the aperture of the imaging lens 12, theobtained microlens images can also be subjected to a three-dimensionalimage processing operation using the parallaxes.

(Relationship Between Variation of Microlens Imaging Magnification andOptical Resolution Depending on Distance to Object) (Difference in MLImaging Magnification Variation Between a Real Image Optical System anda Virtual Image Optical System)

Referring now to FIGS. 6A through 6C, the variation of the microlensimaging magnification with respect to the distance from the imaging lensto an object in the optical system of this embodiment is described.

According to the equation (1), as the distance A from the imaging lensto the object becomes shorter, the imaging length B of the imaging lens12 becomes longer, and as a result, the virtual imaging plane 70 movesfurther away from the microlenses 14 a. Therefore, the microlens imagingmagnification becomes lower according to the following equation:

N=D/(B−E)

On the other hand, as the distance A from the imaging lens to the objectbecomes longer, the imaging length B of the imaging lens 12 becomesshorter, and approaches the focal length f of the imaging lens 12. As aresult, the virtual imaging plane 70 becomes closer to the plane of themicrolenses, and the microlens imaging magnification N becomes higher.

When the object is at an infinite distance, the imaging length B becomesequal to the focal length f of the imaging lens. Therefore, when theobject is at an infinite distance, the imaging magnification N∞ isexpressed by the following equation:

N∞=D/(f−E)

The graph shown in FIG. 6C shows the relationship between the distance Afrom the imaging lens 12 to the object and the imaging magnification (orthe image reduction rate) of the microlenses 14 a when the imagingmagnification N∞ is 0.5, and the focal length f of the imaging lens 12is 5.0 mm. As the object becomes closer to the imaging lens 12, thevirtual imaging plane 70 moves further away from the microlenses 14 a,and the microlens imaging magnification N becomes lower.

The optical resolution degradation characteristics of images formed atthe imaging magnification N by the microlenses 14 a are now described.

When the reduction rate of the microlenses 14 a is N, the opticalresolution is degraded in proportion to the square of the imagereduction rate N for the following reason. The optical samplingfrequency is normally the reciprocal of the pixel pitch. In a case whereuniform-pitch pixels are used, images reduced with respect to the samesampling size are sampled. Therefore, the vertical pitch and thetraverse pitch both become N times larger. Through the multiplication ofthe vertical pitch and the traverse pitch, the degradation is expressedas N2.

In this embodiment, with respect to microlens images, an image pickupoperation needs to be performed at least twice on each object in anoverlapping manner. Since the microlens image reduction rate N at whichan image pickup operation can be performed at least twice is 0.5 orlower, the microlens image reduction rate N is preferably 0.5 or lower.The theoretically highest optical resolution is {the number of sensorpixels×(0.5)²} or lower.

(Method of Re-Forming Two-Dimensional Visible Image)

Referring now to FIGS. 7A through 7C, a method of re-forming atwo-dimensional image having no overlapping portions from microlensimages formed by picking up images of the same object in an overlappingmanner is described.

In this example case, there are three adjacent microlenses, and thosemicrolenses form microlens images 91 a, 91 b, and 91 c, respectively, onthe surface of the imaging element 16, as shown in FIG. 7B. To form suchmicrolens images without any overlapping portions, the F-number of theimaging lens 12 should be nearly equal to the F-number of themicrolenses. The fields of view in which the microlens images 91 a, 91b, and 91 c are formed are equivalent to a field of view 93 a, a fieldof view 93 b, and a field of view 93 c on the visual imaging plane 70,and those fields of view 93 a through 93 c on the visual imaging plane70 are overlapping areas as shown in FIG. 7C. In the example caseillustrated in FIGS. 7B and 7C, the image reduction rate is 0.5. Aftereach field of view becomes 0.5 time larger, an image pickup operation isperformed at least twice on each object point. Where the magnification Nis 0.5, the image formed on the virtual imaging plane 70 can bereproduced by enlarging each microlens image at a magnification of 1/Nor a magnification of 2.

To determine the image reduction rate N from the microlens images afterimage pickup operations, the following equation (12) that is based onthe relationship expressed by the equation (6) is used:

$\begin{matrix}{N = {\frac{D}{C} = \frac{\Delta}{L_{ML}}}} & (12)\end{matrix}$

Since the pitch LML of the microlenses is already known, the imagereduction rate N can be determined by calculating the shift length Δ ofthe same object from images.

To calculate the shift length Δ of the same object from images, an imagematching operation is performed on adjacent microlens images. In theimage matching operation, a known template matching method can be usedto check the similarity and difference between two images, for example.To calculate the shift length Δ with higher precision, the similaritiesand differences obtained with respect to the respective pixels areinterpolated by a continuous fitting function or the like, and sub-pixelpositions that define the maximum and the minimum of the fittingfunction are determined. In this manner, the shift length Δ can becalculated with even higher precision.

(Method of Obtaining Object Depth-Direction Distance Information)

As described with reference to FIG. 6C, there is a correlation betweenthe distance A from the imaging lens to an object and the imagereduction rate N. Therefore, when the shift length Δ is calculated fromthe equation (12) by the above described method, the image reductionrate N is determined, as the pitch LML of the microlenses is a fixeddesigned value. As a result, the distance A from the imaging lens to theobject can be determined.

As described so far, according to the first embodiment, an increase incamera height can be reduced.

Further, images with parallaxes can be obtained through microlenses.Accordingly, various three-dimensional image processing operations usingparallax images can be performed.

Second Embodiment

Referring now to FIG. 8, a solid-state imaging device (a camera module)according to a second embodiment is described. FIG. 8 is across-sectional view of the solid-state imaging device 1A of the secondembodiment. The solid-state imaging device 1A of the second embodimentis the same as the solid-state imaging device 1 of the first embodiment,except that the visible light transmissive substrate 40 is replaced withan optical filter 41. That is, the lens holder 64 supporting the imaginglens 12 and the lens barrel 62 is joined onto the peripheral region ofthe optical filter 41. The optical filter 41 is a substrate made of asubstrate material that transmits visible light (the R, G, and Bwavelengths) and cuts off unnecessary near-infrared light, and themicrolens array 14 is formed on one side of the substrate. The substratematerial of the optical filter 41 may be a material that absorbsunnecessary near-infrared light, or may have a multilayer film or asingle-layer film that reflects near-infrared light.

The optical filter substrate 41 having the microlens array 14 formedthereon is positioned so that the microlenses face the imaging element16.

In the second embodiment, an increase in camera height can be reduced,and images with parallaxes can be obtained through microlenses, as inthe first embodiment. Accordingly, various three-dimensional imageprocessing operations using parallax images can also be performed.

The solid-state imaging devices of the first and second embodiments canbe applied not only to still cameras but also to a portable informationterminal 200 shown in FIG. 9. The portable information terminal shown inFIG. 9 is merely an example, and reference numeral 10 denotes theimaging module of the solid-state imaging device according to the firstor second embodiment.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein can be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein can be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

1. An imaging device comprising: an imaging element including aplurality of pixels; a lens; and a plurality of microlenses, an imagingplane of the lens being located further away from the lens than theimaging element when an object is located at an infinite distance. 2.The device according to claim 1, wherein the microlenses are semi convexlenses having a convex shape toward the imaging element.
 3. The deviceaccording to claim 1, further comprising color filters that are providedfor the respective pixels.
 4. The device according to claim 1, whereinthe microlenses reduce and re-form an image to be formed on the imagingplane respectively.
 5. The device according to claim 4, wherein thereduction rate of the microlenses is 0.5 or lower when the object islocated at the infinite distance.
 6. The device according to claim 1,wherein the microlenses are arranged in a hexagonal closed-packed array.7. The device according to claim 1, wherein the imaging element isprovided on a semiconductor substrate, and the microlenses are providedon a substrate that absorbs or reflects infrared light whiletransmitting visible light.
 8. The device according to claim 7, furthercomprising a spacer provided between the semiconductor substrate and thesubstrate on which the imaging element is provided, the spacer locatedaround the periphery of the imaging element.
 9. The device according toclaim 1, further comprising an image signal processor that calculates ashift length of the same object from microlens images and determines adistance from the lens to the object based on the shift length.
 10. Thedevice according to claim 9, wherein the shift length is calculated byperforming an image matching operation.
 11. The device according toclaim 9, wherein the image signal processor determines an imagereduction rate based on the shift length and determines the distancefrom the lens to the object based on the image reduction rate.
 12. Aportable information terminal comprising an imaging device, the imagingdevice comprising: an imaging element including a plurality of pixels; alens; and a plurality of microlenses, an imaging plane of the lens beinglocated further away from the lens than the imaging element when anobject is located at an infinite distance.
 13. The terminal according toclaim 12, wherein the microlenses are semi convex lenses having a convexshape toward the imaging element.
 14. The terminal according to claim12, further comprising color filters that are provided for therespective pixels.
 15. The terminal according to claim 12, wherein themicrolenses reduce and re-form an image to be formed on the imagingplane respectively.
 16. The terminal according to claim 15, wherein thereduction rate of the microlenses is 0.5 or lower when the object islocated at the infinite distance.
 17. The terminal according to claim12, wherein the microlenses are arranged in a hexagonal closed-packedarray.
 18. The terminal according to claim 12, wherein the imagingelement is provided on a semiconductor substrate, and the microlensesare provided on a substrate that absorbs or reflects infrared lightwhile transmitting visible light.
 19. The terminal according to claim18, further comprising a spacer provided between the semiconductorsubstrate and the substrate on which the imaging element is provided,the spacer located around the periphery of the imaging element.
 20. Theterminal according to claim 12, further comprising an image signalprocessor that calculates a shift length of the same object frommicrolens images and determines a distance from the lens to the objectbased on the shift length.
 21. The terminal according to claim 20,wherein the shift length is calculated by performing an image matchingoperation.
 22. The terminal according to claim 20, wherein the imagesignal processor determines an image reduction rate based on the shiftlength and determines the distance from the lens to the object based onthe image reduction rate.